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ABSTRACT 

This  exposition  presents  a method  for  incorporating  a technique 
known  as  ''splitting  the  bump"  within  an  elimination  form  relnverslon 
algorithm.  This  procedure  is  designed  to  reduce  flll-ln  during  rein- 
version  and  should  Improve  the  efficiency  of  linear  programming  systems 
which  already  use  the  superior  elimination  form  of  the  Inverse. 
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I.  limODUCTlON 


Current  production  lineer  progreaning  systems  are  designed  to 
handle  problems  with  8000  to  16,000  rows  and  1000  row  problems  are 
considered  to  be  medium  sized  (see  [10,  17].  Fortunately  real  pro- 
blems tend  to  be  sparse  (the  density  of  the  constraint  matrix  is 
often  less  than  IZ  - see  [1,  2,  9,  14,  15]).  Hence,  a basis  for  a 
1000  row  problem  may  have  only  (1000)  (1000)  (IZ)  ••  10,000  nonzero 
entries.  However,  the  inverse  (which  is  required  for  the  revised 
simplex  algorithm)  may  be  quite  dense  having  almost  a million  nonzero 
elements.  Consequently,  one  of  the  most  Important  design  considerations 
for  a computer  Implementation  of  the  revised  simplex  algorithm  for  general 
linear  programs  is  the  technique  used  to  maintain  and  update  the  basis 
inverse. 

In  order  to  minimize  the  storage  required  to  Implement  this  algo- 
rithm, production  linear  programming  systems  maintain  the  inverse  of 
the  basis  in  either  product  or  elimination  form  (see  [1,  3,  8,  12,  17]). 
Computationally  the  Inverse  is  stored  as  a sequence  of  vectors,  known 
as  &ta  victofiA,  and  the  complete  list  of  the  eta's  is  called  the  ETA 
fiJte..  Bach  basis  change  (pivot)  results  in  appending  at  least  one  eta 
vector  to  the  ETA  File.  Since  both  the  time  per  pivot  end  numerical 
error  Increase  as  the  length  of  the  ETA  File  Increases,  it  is  necessary 
to  periodically  obtain  a new  factorization  of  the  basis  Inverse.  This 
process  of  obtaining  a new  factorization  is  called  KeJjwzAZ^n.  The 
objective  of  a relnversion  algorithm  is  to  obtain  a factored  Inverse 
(i.e.,  ETA  File)  in  which  the  sparsity  characteristics  of  the  original 
basis  are  preserved  as  much  as  possible. 
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The  slnplest  relnverslon  technique  for  e given  m-coluan  basis 
can  be  thought  of  as  successively  reducing  the  basis  to  an  Identity 
■atrlx  via  m pivot  operations.  The  matrices  vhlch  accomplish  this 
reduction  constitute  the  ETA  File,  often  refered  to  as  a product  form 
factorization.  Out  of  this  slaple  approach,  relnverslon  techniques 
have  evolved  which  attempt  to  obtain  a sparse  factorization  by  selection 
of  pivot  positions.  Involving  a reordering  of  the  coluans  of  the  basis 
and  a conceptual  reordering  of  the  rows.  The  row  and  column  pivot 
ordering  employed  in  relnverslon  Is  usually  called  the  pivot  agenda.. 

Markowitz  [11]  pioneered  the  work  in  this  area.  Another  technique 
(see  [8]  and  [13])  which  has  been  used  in  conjunction  with  the  relnverslon 
algorithms  Is  a technique  known  as  Aptitting  the  bump.  All  these  Ideas 
were  directed  toward  developing  a sparse  representation  using  the  product 
form  of  the  Inverse.  Recently,  however,  coi^utatlonal  evidence  Indicates 
that  the  elimination  form  factorization  Is  superior  to  the  product  fora 
factorization  (with  respect  to  both  storage  requirements  and  speed  (see  [2]). 
The  Hellerman-Rarlck  algorithm  can  be  easily  modified  to  accommodate 
an  elimination  form  relnverslon,  but  It  Is  not  obvious  how  one  may  im- 
plement the  technique  known  as  "splitting  the  bump"  and  still  make  It 
possible  to  maintain  the  elimination  form  of  the  inverse  during  later 
simplex  Iterations.  The  objective  of  this  exposition  is  to  present  a 
relnverslon  algorithm  in  which  one  may  achieve  some  of  the  benefits  of 

the  "splitting  the  bump  technique"  when  using  the  elimination  form 
of  the  Inverse. 
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For  this  exposition,  the  coluan  of  the  a x n aetrlx  B is  denoted 
by  B(l).  The  symbol  e^  denotes  the  a-coaponent  coluan  vector  having 
i^^  component  one  and  all  other  coBq>onents  aero.  The  syabol  n denotes 
an  B»-component  coluan  vector  and  denotes  the  1^^  component  of  this  vector. 
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II.  FACTORIZATION  ALGORITHMS 

Let  B be  any  m by  m nonslngular  matrix.  In  this  section  we  present 
two  algorithms  for  obtaining  a factorization  of  B The  first  algorithm 
produces  a product  form  factorization  which  corresponds  to  the  method 
for  solving  a system  of  linear  equations  known  as  Gauss— Jordan  reduction 
while  the  second  algorithm  produces  an  elimination  form  factorization 
which  corresponds  to  a Gauss  reduction  (see  [4]). 

By  row  and  column  interchanges,  B may  be  placed  In  the  following 

form: 


1 3 

where  B and  B are  lower  triangular  matrices  with  nonzeroes  on  their 

2 

diagonals.  We  assume  that  If  B Is  nonvacuous,  every  row  and  column 

2 

has  at  least  two  nonzero  entries,  so  that  no  rearrangement  of  B can 

13  2 

expand  the  size  of  B or  B . B Is  called  the  bump,  nCAaCt,  or  hiOAt 
iZCtion.  We  further  require  the  heart  section  to  assume  the  following 
form: 


W 


where  G 's  are  either  vacuous  or  lower  triangular  trlth  nonzeroes  on 
the  diagonal.  The  only  partitions  In  B having  columns  with  nonzeroes 

V 

above  the  diagonal  are  the  F 's  which  are  called  e.Ktt>incU.  butnpi.  The 
coluans  extending  above  the  diagonal  are  called  SpiktS  or  6pZke.  cotunni.  . 
An  external  bump  Is  characterized  as  follows: 

(1)  the  last  coltimn  of  an  external  bump  will 
be  a spike  with  a nonzero  lying  In  the 
topmost  row  of  the  external  bump,  and 
(11)  the  nonspike  columns  have  nonzero  diagonal 
elements. 

The  algorithms  of  Hellerman  and  Rarlck  [6,  7]  produce  such  a structure 
for  any  nonsingular  matrix  B,  and  we  shall  call  a matrix  having  this 
structure  an  HR  matrix. 

Tha  product  form  factorization  algorithm  for  an  HR  matrix  Is 
given  as  follows: 

ALG  U PROVUCT  FORM  FACTORIZATION  m A HR  MATRIX 

0.  Jnitltttization 

Sat  1^1  and  6 B(l) . 
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1.  Obtain  Mew  ETA 


1/e^,  for  k - 1 


Set 


and  set 


E^(k) 


n,  for  k ■ 1 


e , otherwise. 


(Note:  only  n 1 need  be  saved  In  the  ETA  File) 
2.  Tut  Tofi  TcAmination 


If  i > m,  terminate,  otherwise,  1 1 1. 

3.  Tut  Foa.  Spike 

If  B(l)  Is  not  a spike,  set  $ ^ B(l)  and  go  to  1. 

4 . Spike  Update 


Let  B(k)  correspond  to  the  first  column  of  the  external  bump 
containing  B(i).  Set  6 ■«-  ^ •••  E*S(1). 


5.  Swap  Spiku  Ji  Pivot  Element  Equals  ZeAo 


If  3^  1*  0,  go  to  1;  otherwise,  there  is  some  spike  B(J)  in  the 


,th 


same  external  bump  having  J > i such  that  the  i cooponent  of 
E^  ^ •••  E*^B(j)  is  nonzero.  Set  6 E^”^  •••  E*S(J),  Interchange 

B(l)  and  B(j)  and  go  to  1. 


In  practice,  the  test  for  a zero  pivot  element  in  atep  5 is  usually 
replaced  by  a tolerance  test.  Let  TOL  denote  the  tolerance  to  be  uaed  in 
test.  If  I ^ TOL,  then  8^^  is  treated  as  if  it  is  zero.  Discuss* 
ions  of  tolerences  msy  be  found  in  [12,  15,  16].  Similar  tolerance  tests 
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! 
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I 
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would  ordinarily  be  Incorporated  In  the  other  algorithms  to  be  presented 
in  this  exposition.  To  simplify  the  presentation  they  have  been  omitted  here. 

Justification  for  ALG  1 is  given  in  [5].  At  termination,  B ^ ■ 
e"e“-^  ...  e\  Furthermore,  we  see  that  for  nonspike  columns,  each 
6 and  consequently  each  n has  exactly  the  same  number  of  nonzeroes  as  the 

corresponding  column  of  B.  However,  the  n for  a spike  column  may  have 
a higher  density  than  the  original  column  of  B.  The  phenomena  of  an 
r)  having  a higher  density  than  the  corresponding  column  of  B is  known 
as 

It  is  well  known  that  fill-in  may  be  reduced  by  applying  an  elimi- 
nation form  factorization  rather  than  the  product  form  (see  [1]).  The 
elimination  form  algorithm  for  an  HR  matrix  is  given  as  follows: 

ALG  2:  ELIMIMATIOM  FGR/K  FACTORIZATION  FOR  A HR  AUTRIX 

0.  InitiAtizcuUon 


1. 


Set  1 ■«-  1,  r 0,  and  $ *■  B(l). 
Obtain  New  Louka  Eta 

r 1/6^,  for  k • 


Set 


and  set  L 


^ 0,  otherwise , 

j f for  k • 1 

(k)  - / 

e*^,  otherwise 


1 

> i 


(Hots:  only  n and  1 need  be  saved  in  the  Eta  File  for  the  lower  factors) 
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2.  TeA-t  Foa  TzAmunatoon 


If  i • m,  terminate;  otherwise,  i i + 1. 

3.  Tut  foA  SptMz 

If  B(i)  is  not  a spike,  set  6 B(i)  and  go  to  1. 

4.  Sptke  Update 

Let  B(k)  correspond  to  the  first  colinm  of  the  external  bump 
containing  B(l).  Set  •••  L^Sd). 

5.  Swap  SfUku  li  Ptvot  Etzmcnt  Equate  ZeAo 

If  0,  go  to  6;  otherwise,  there  is  some  spike  B(j)  in 

the  same  external  bump  having  j > i such  that  the  i^  component 
of  •••  L^B(j)  is  nonzero.  Set  B *•*  L "BCj)  and  inter 

change  B(i)  and  B(j). 

6.  ObtcUn  Hew  UppeA  Eta 

Set  r r + 1 


Set 


set 


u’^Ck) 


{ 


1,  for  k - i 
-6^^,  for  k < i 

0,  otherwise, 
n,  for  k ■ i 

k 

e , otherwise. 


and  go  to  1. 

(Note:  only  n and  i need  be  saved  in  the  Eta  File  for  the  upper  factors) 
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• • • 


L^,  where  the  upper 


At  the  termlnatloQ  of  ALG  2 , B ^ • * ' U^L* 

eta's  are  upper  triangular  and  the  lower  eta's  are  lower  triangular, 
with  ALG  1 fill- In  has  been  restricted  to  the  eta's  corresponding  to 
columns. 


As 


spike 


III.  SPLITTIUG  THE  BUMP 


In  an  attempt  to  reduce  the  fill-in  which  occurs  in  spike  columns 
during  a reinversion  using  ALG  1,  a variation  of  the  elimination  form 
factorization  algorithm  (attributed  to  Martin  Beale,  see  [13])  has  been 
used  by  some  practitioners.  This  technique  has  been  called  "splitting 
the  bump"  after  its  treatment  of  external  bump  columns. 

Consider  a set  of  columns  of  the  basis  corresponding  to  an  external 
bump , say 


}.  ^ 

where  F is  an  external  bump.  Suppose  F contains  q spikes.  If  we  apply 
the  standard  product  form  algorithm,  we  obtain  a sets  of  eta's  such  that 


The  eta's  corresponding  to  the  q spikes  may  incur  fill-in.  If  we  split 
the  bump,  the  fill-in  can  be  restricted  to  the  n rows  corresponding  to 
F;  hence,  we  may  avoid  fill-in  in  the  last  p rows.  Since  p may  be  much 


larger  Chan  n,  the  savings  could  be  substantial.  The  price  which  must 
be  paid  to  attain  this  reduction  in  fill-in  is  that  each  external  bump 
will  require  2n  q eta's  rather  than  n.  The  savings  in  fill-in  is 


typically  so  great  that  it  offsets  the  additional  storage  which  must  be 
given  up  for  the  additional  eta's. 

We  first  develop  a set  of  n eta's  such  that 


where  r has  -fl's  on  the  diagonal  and  zeroes  below  the  diagonal.  The 
only  columns  having  nonzeroes  above  Che  diagonal  are  spike  columns. 
Next  we  zero  out  the  spikes  in  by  developing  q eta's  such  that 


Finally,  we  zero  out  the  H via  the  addition  of  n more  eta's.  That  is. 
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1 

i 

( 

The  product  form  algorithm  incorporating  the  "splitting  the  buiiq>"  ] 

technique  is  given  as  follows: 

ALG  3:  PRODUCT  ¥ORU  FACTORIZATION  FOR  A HR  MATRIX  INCLUDING  SPLITTING 
THE  BUMP 

0.  Jn^U-ializAtion 

Set  1 •«-  1,  j 1,  and  $ ■*-  B(l).  If  B(l)  is  in  an  external  bump,  j 

to  to  4. 

1.  Ob-tcun  New  LoweA.  Eta  ^ 


5.  Obtain  LoueA  Eta 


Set 


1/6^,  for  k - 1 

1 < k ^ t 
0,  otherwise, 


and  set 


E^Ck) 


Ti,  for  k ■ i 


e , otherwise. 


Set  j j + 1. 

6.  Tc^t  Foa  End  Oi  Bump 

If  i • t,  go  to  10;  otherwise,  1 1 + 1. 

7.  Tut  Foa  Spike 

If  B(l)  Is  not  a spike,  set  B *■  B(i)  and  go  to  5. 

8.  Spike  Update 

Set  B * E-^"^  •••  eSu). 

9.  SMXip  Spikes  li  Pivot  Etement  Equate  ZeAO 

If  8^  y 0,  go  to  5;  otherwise,  there  is  some  spike  B(r)  In  the 
saae  external  bunp  having  r > 1 such  that  the  1^^  consonant  of 

...  E^B(r)  Is  nonsero.  Set  8 E'^"’^  •••  E^B(r),  interchange 

B(r)  and  B(l)  and  go  to  5. 

10.  Obtain  UppeA  Eta 

r 1,  for  k • 1 

Set  \ ( "^k*  ^ 

^ 0,  otherwise. 
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and  aet  S'*  (k) 


r r,,  for  k ■ i 
otherwise. 


Set  j J + 1 and  1 1 - 1. 

11.  Tut  foK  BtgintUng  Oi  Bunp 

If  i - 8,  set  B *►  B(i)  and  go  to  13. 

12.  Tut  Tofi  Spike. 

If  B(i)  is  not  a spike,  set  i 1 - 1 and  go  to  11;  otherwise, 
set  B •••  E*'B(i)  and  go  to  10. 


13.  ObtcUn  LomA  Eta. 


and  set 


E^(k) 


1,  for  k - i 


/ -Bj^,  for  k > t 


0,  otherwise, 


n,  for  k ■ i 

k 

e , otherwise. 


Set  J •’■J  + !• 


14.  TtU  Foa  End  Oi  Bump 


If  i ■ t,  go  to  2;  otherwise  i i + 1,  set  B B(i),  and  go 


to  13. 


•ca's  ara  aithar  upper  triangular  or  lower  triangular,  but  they  are 


Bacall  that  ALG  2 produces  a set  of  lower  triangular  factors 


followed  by  a sat  of  upper  triangular  factors.  If  one  Incorporates 


the  "splitting  the  buaq>"  technique  Into  ALG  2 and  apply  It  to  an  HR 


Crlx,  the  lower  and  upper  factors  become  Intermingled.  Once  the 


factors  have  become  Intermingled,  tw  may  no  longer  use  the  Important 


algorithm  of  Forrest  and  Tomlin  [2]  to  maintain  the  elimination  form 
We  now  show  how  one  msy  achieve  a partial  "splitting  of  the  bump" 


while  simultaneously  maintaining  a partitioning  of  the  upper  and  lower 


Recall  that  an  HR  matrix  takes  the  following  form 


ALG  3 (Product  Factorisation  With  Splitting  The  Bump)  eliminates  all 


fill- In  In  B . By  a rearrangement  of  rows  and  columns,  B may  be  placed 


In  the  following  form: 


where  end  ere  row  permutetlons  of  B^  and  B^,  reepectively,  end 
-3  3 

B • le  e row  end  colunn  penmitetlon  of  B . Applying  e verletion  of 
ALG  2 to  B ellninatee  all  flll-ln  in  B^  while  representing  B ^ as  a 
product  of  upper  factors  followed  by  a product  of  lower  factors.  The 
details  are  given  in  the  following  algorltta. 

ALG  4t  ELIHUNATION  FORM  FACTORIZATION  FOR  A MOPIFIEP  HR  MATRIX 
0.  InitiatLzcuUon 


Set  1 ■«-  u + 1 and  6 B(l).  If  v ■ 0,  go  to  3 

1.  ObteUn  Nenf  LotoM.  Eta 


Set 


1/B^,  for  k - 1 
for  k > 1 
0,  otherwise  , 


and  set  L^(k) 


r T),  for  k ■ i 
^ e^,  otherwise . 
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r 


r 

i 


! 

i 


2.  Te^t  Foa  End  0^  Section  2 

S«t  i i + 1 and  8 *■  5(1).  If  i ^ u + v,  go  to  1.  If  w • 0, 
to  to  8. 


3.  Obtain  New  Louen  Eta 


Sat  \ ♦ 


1/8^,  for  k - 1 
^ k > 1 

0 , othcxvlsc , 


and  aet 


L^(k) 


n,  for  k - 1 
a^,  otharvisa. 


i 

i 


I 


4.  Te4t  Foa  End  0{  Section  5 

If  1 > a,  go  to  8;  otherwise,  1 <«'  1 *f  1. 

5.  Tut  Foa  Spike 

If  5(1)  la  not  a spike,  aet  8 5(1)  and  go  to  3. 

6.  Spike  Update 

Sat  8 •••  l“^4(1). 


I 

I 


I 

f 

t 

i 


7.  Swop  Spiku  J{  Pivot  Etenent  Equate  leAo 

It  8^  ^ 0,  go  to  3;  otharvisa,  there  la  soae  spike  B(J)  In  the 
saaa  axtamal  buap  having  J > 1 such  that  the  1^^  coaponant  of 
...  L“*^H(j)  la  nonsaro.  Sat  8 •••  L"^^5(j), 

Interchange  5(1)  and  B(J)  and  go  to  3. 
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I ' 

1 
I 


8.  ObtcUn  Hew  UppeA.  Eta. 


1,  for  k - i 


Set 


for  k < i 


0,  othcrvlw. 


and  set 


U^(k) 


r Tit  for  k ■ 1 
^ e*^,  otherwise. 


9.  Check  Eok  End  Oi  Se.ctlon  2 


Set  Ifi^O,  terminate. 

If  1 * u,  set  $ B(i)  and  go  to  12. 


10.  Set  Cotxmn 


If  i > u + r,  set  B L •< 
set  B B(i).  Go  to  8. 


j^u+v+lg^j,  otherwise. 


U.  ObtcUn  Hew  Uppen  Eta 


1/B^t  for  k - 1 


Set 


-Bj^/Bn  for  k < i 
0,  otherwise. 


and  set 


0^(k) 


r Tit  for  k ■ 1 
1 e^,  otherwise. 


12.  Check  Foa  TeAmination 


If  i > It  terminate;  otherwise  .B  B(l)  and  go  to  11. 


?9  . 


At  the  tcnlnatlon  of  ALG  4,  B •••  lA,”  and  the  fill-in 

2 

hae  been  restricted  to  B . Hence,  ALG  4 gains  some  of  the  benefits 
of  the  "splitting  the  bunp"  technique  while  asintaining  a partition- 
ing of  the  upper  and  lower  factors.  The  benefits  are  not  as  great 
as  with  ordinary  "buwp  splitting"  since  each  individual  external  buap 
is  split,  whereas  here  the  split  is  with  respect  to  the  entire  heart 
section. 

A variation  of  ALG  4 has  also  been  used  by  Toalin  [15].  Our 
contribution  is  that  we  have  tied  together  the  ideas  of  "splitting 
the  buaq>"  in  both  the  product  and  elimination  fora  factorisations 
and  we  have  explicitly  indicated  via  ALG's  3 and  4 how  these  may  be 
iapleaented. 
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